SurfaceCast: Ubiquitous, Cross-Device Surface Sharing

Real-time online interaction is the norm today. Tabletops and other dedicated interactive surface devices with direct input and tangible interaction can enhance remote collaboration, and open up new interaction scenarios based on mixed physical/virtual components. However, they are only available to a small subset of users, as they usually require identical bespoke hardware for every participant, are complex to setup, and need custom scenario-specific applications. We present SurfaceCast, a software toolkit designed to merge multiple distributed, heterogeneous end-user devices into a single, shared mixed-reality surface. Supported devices include regular desktop and laptop computers, tablets, and mixed-reality headsets, as well as projector-camera setups and dedicated interactive tabletop systems. This device-agnostic approach provides a fundamental building block for exploration of a far wider range of usage scenarios than previously feasible, including future clients using our provided API. In this paper, we discuss the software architecture of SurfaceCast, present a formative user study and a quantitative performance analysis of our framework, and introduce five example application scenarios which we enhance through the multi-user and multi-device features of the framework. Our results show that the hardware- and content-agnostic architecture of SurfaceCast can run on a wide variety of devices with sufficient performance and fidelity for real-time interaction

Using an Infrared Pen as an Input Device for Projected Augmented Reality Tabletops

Interactive tabletops do not only offer a large surface for collaborative interaction. They also offer quick access to digital tools directly at the table - where a large number of everyday activities take place. Tabletops with an embedded display are generally less flexible and more fragile than ordinary massive tabletops. Physical objects on the tabletop occlude the digital content. In contrast, top-down-projected interfaces using an overhead camera-projector system allow for augmenting arbitrary tables and the object lying on them. However, detecting pointing input only via a camera image captured from above requires robustly recognizing whether a finger or pen touches the tabletop or whether it hovers slightly above it. In this paper, we present a solution for reliably tracking a pen on arbitrary tabletop surfaces. The pen emits infrared light via a tip made of optical fiber. A camera captures position and shape of the light point on the surface. Our open-source tracking algorithm combines heuristics and a machine learning model to distinguish between drawing and hovering. A pilot study with 7 participants shows that that this system can be reliably used for drawing and writing on tabletops. However, occlusion by users’ hands can deteriorate tracking of the pen

Demonstrating SurfaceCast:Ubiquitous, Cross-Device Surface Sharing

Real-time online interaction is the norm today. Tabletops and other dedicated interactive surface devices with direct input and tangible interaction can enhance remote collaboration, and open up new interaction scenarios based on mixed physical/virtual components. However, they are only available to a small subset of users, as they usually require identical bespoke hardware for every participant, are complex to setup, and need custom scenario-specific applications. We present SurfaceCast, a software toolkit designed to merge multiple distributed, heterogeneous end-user devices into a single, shared mixed-reality surface. Supported devices include regular desktop and laptop computers, tablets, and mixed-reality headsets, as well as projector-camera setups and dedicated interactive tabletop systems. This device-agnostic approach provides a fundamental building block for exploration of a far wider range of usage scenarios than previously feasible, including future clients using our provided API. In this paper, we present various example application scenarios which we enhance through the multi-user and multi-device features of the framework. Our results show that the hardware- and content-agnostic architecture of SurfaceCast can run on a wide variety of devices with sufficient performance and fidelity for real-time interaction

16. Workshop Be-greifbare Interaktion

Tangible interaction is centered around the manipulation of physical objects and the usage of the own body. It thus involves the environment and the physical context much stronger than purely visual or speech-based interfaces. The wide range of possibilities for integrating sensors and computing systems into the physical environment provides ample design space. The research field of Tangible Interaction investigates this scope scientifically and practically in order to enable meaningful and human-oriented applications. In this workshop, the German Informatics Society (GI) specialist group "Be-greifbare Interaktion" of the Department of Human-Computer Interaction offers a forum for the presentation of scientific discourse and interdisciplinary discussion. Contributions range from theoretical, critical and forward-looking reflections to design work and reports on practical implementations. The workshop opens the discussion to a broader audience of experts in order to disclose current developments and generate new impulses for the research field